Method and apparatus for generating a haptic effect in response to flex gesturing

ABSTRACT

An electronic interactive device having a user interface with a flexible surface, a sensor configured to sense a flex gesture applied to the flexible surface, a haptic device configured to generate a haptic effect in response to the flex gesture, and a controller in signal communication with the sensor and the haptic device. The controller is configured to trigger the haptic device to provide haptic confirmation of the flex gesture and to trigger an application action associated with the flex gesture.

FIELD OF THE INVENTION

The invention relates to a method and apparatus for haptic flexgesturing.

BACKGROUND OF THE INVENTION

Some known electronic displays may be flexible and can mimic propertiesof paper, such as an ability to be bent or rolled, but have limited waysto interface with a user. User inputs may be received through buttons orkeys coupled to a flexible display or through a flex gesture applied tothe flexible display. In response to a flex gesture, a change in animage shown on the display may be provided as the user output.

SUMMARY

According to an aspect of the present invention, there is provided asystem configured to provide a haptic effect through an electronicinteractive device. The electronic interactive device may comprise auser interface having a flexible surface and a sensor configured tosense a flex gesture applied to the flexible surface. The electronicinteractive device may further comprise a haptic device and acontroller. The haptic device may be configured to generate a hapticeffect in response to the flex gesture. The controller may be in signalcommunication with the sensor and the haptic device, and may beconfigured to trigger the haptic device to provide haptic confirmationof the flex gesture and to trigger an application action associated withthe flex gesture.

In some embodiments, the flex gesture sensed by the sensor may comprisebending of the flexible surface or twisting of the flexible surface.

In some embodiments, the controller may be configured to trigger thehaptic device to provide a first haptic effect in response to thebending and to trigger the haptic device to provide a second hapticeffect in response to the twisting. In an embodiment, the sensor may beconfigured to sense the flex gesture in a first direction and to sensethe flex gesture in an opposite direction. The controller may beconfigured to trigger a first application action in response to thesensor sensing the flex gesture in the first direction and to trigger asecond application action in response to the sensor sensing the flexgesture in the opposite direction. For example, the first applicationaction and the second application action may be actions of an on-offtoggle action, a select-deselect toggle action, an up-down toggleaction, a left-right toggle action, a forward-reverse toggle action, azoom-in-zoom-out toggle action, or any combination thereof.

In some embodiments, the controller may be configured to trigger theapplication action by outputting a signal associated with the flexgesture to a computing device configured to execute an applicationassociated with the application action. A wireless communication device,for example, may be part of the electronic interactive device and may beconfigured to transmit the signal to the computing device. In anembodiment, the user interface may comprise a flexible display screen.The controller may be configured to trigger the application action bypresenting the application action on the display screen.

In some embodiments, the electronic interactive device may comprise asecond sensor configured to sense a touch gesture applied to theflexible surface. The controller may be configured to trigger theapplication action only if the second sensor has sensed the touchgesture and the sensor has sensed the flex gesture.

In some embodiments, the flexible surface of the electronic interactivedevice may be located at an edge of the electronic interactive device.In an embodiment, the sensor may comprise a strain gauge, apiezoelectric transducer, an accelerometer, a gyroscope, or anycombination thereof.

These and other aspects, features, and characteristics of the presentinvention, as well as the methods of operation and functions of therelated elements of structure and the combination of parts and economiesof manufacture, will become more apparent upon consideration of thefollowing description and the appended claims with reference to theaccompanying drawings, all of which form a part of this specification,wherein like reference numerals designate corresponding parts in thevarious figures. It is to be expressly understood, however, that thedrawings are for the purpose of illustration and description only andare not intended as a definition of the limits of the invention. As usedin the specification and in the claims, the singular form of “a”, “an”,and “the” include plural referents unless the context clearly dictatesotherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates an apparatus in accordance with anembodiment of the invention.

FIG. 2 schematically illustrates components of the apparatus of FIG. 1.

FIG. 3 schematically illustrates flex gestures being applied to theapparatus of FIG. 1.

FIG. 3A illustrates a cross-section of the apparatus of FIG. 3 takenalong line 3A-3A.

FIG. 4 schematically illustrates a flex gesture being applied to theapparatus of FIG. 1.

FIG. 4A illustrates a cross-section of the apparatus of FIG. 4 takenalong line 4A-4A.

FIG. 4B illustrates a cross-section of the apparatus of FIG. 4 takenalong line 4B-4B.

FIG. 5 schematically illustrates a flex gesture being applied to theapparatus of FIG. 1.

FIG. 5A illustrates a cross-section of the apparatus of FIG. 5 takenalong line 5A-5A.

FIG. 5B illustrates a cross-section of the apparatus of FIG. 5 takenalong line 5B-5B.

FIG. 6 schematically illustrates a flex gesture being applied to theapparatus of FIG. 1

FIG. 6A illustrates a cross-section of the apparatus of FIG. 6 takenalong line 6A-6A.

FIG. 6B illustrates a cross-section of the apparatus of FIG. 6 takenalong line 6B-6B.

FIG. 7 schematically illustrates a flex gesture being applied to theapparatus of FIG. 1.

FIG. 7A illustrates a cross-section of the apparatus of FIG. 7 takenalong line 7A-7A.

FIG. 7B illustrates a cross-section of the apparatus of FIG. 7 takenalong line 7B-7B.

DETAILED DESCRIPTION

FIG. 1 illustrates an embodiment of an electronic interactive device 100that may provide a haptic effect and that may trigger an applicationaction in response to a flex gesture applied to a flexible surface 110of the device. A flex gesture generally refers to a gross deformation ofa surface as opposed to slight flexures that may occur when a touchscreen surface, for example, is pressed. The flex gesture may includebending, twisting, rolling, or any combination thereof, applied to thesurface. An application action may include any response that anapplication or other program may have to a user input. The flex gesturemay thus supplement or replace other forms of user input, such aspressing a button, pulling a trigger, or clicking a mouse. In someembodiments, electronic interactive device 100 may further include userinput devices 122, 124, 126, and 128 and screen 130. Device 100 thusprovides a user interface that can be used as part of a game controlleror any other remote control (e.g., TV remote, steering wheel gameremote), an eReader (e.g., electronic newspaper, magazine, map, orbook), an interactive toy, a tablet gaming or mobile multimedia device,or any other electronic interactive device.

Flexible surface 110 may be formed from one or more flexible materialssuch as a plastic capable of undergoing a gross, elastic deformation asa result of a flex gesture applied by a user. In some embodiments,flexible surface 110 may cover only part of electronic interactivedevice 100, which may have one or more rigid portions. For example, acenter portion of device 100 may embed a rigid PCB board while an edgeor a corner of device 100 may comprise flexible surface 110. In someembodiments, flexible surface 110 may constitute the entire surface ofdevice 100. In some embodiments, flexible surface 110 may have anadjustable stiffness. For example, the surface may comprise materialhaving a stiffness that is affected by electrical current or heat. Acontroller of device 100 may cause current or heat to be applied to thematerial to control stiffness of flexible surface 110.

User input devices 122, 124, 126, and 128 may include a button, atrigger, a joystick, a knob, a switch, a slide bar, a touch pad, or anyother user input device. One or more of the user input devices may becoupled to flexible surface 110 or may be coupled to a rigid portion, ifany, of device 100. One or more of the user input devices may be amechanical user input device or may be a touch-sensitive device. Forexample, user input device 126 may comprise four mechanical buttonswhile user input device 122 may comprise two touchpads configured tomimic a mechanical joystick. In that or another example, user inputdevice 124 may comprise four buttons formed from touch-sensitive areasof device 100, such as touch-sensitive areas of flexible surface 110.User input device 128 may comprise two mechanical triggers coupled to afront side of device 100, or may comprise two touch-sensitive buttonsformed on the front side.

In some embodiments, screen 130 may also be a user input device, such asa touch screen. In some embodiments, screen 130 may provide only useroutput. Screen 130 may be part of a rigid portion, if any, of electronicinteractive device 100, or may be operatively coupled to flexiblesurface 110 or form part of flexible surface 110. For example, screen130 may be embedded within surface 110 or may be mounted on surface 110.In some embodiments, flexible screen 130 may be formed from a Gyriconsheet, which is a type of electronic paper developed at Xerox PARC®(Palo Alto Research Center). The Gyricon sheet has similar physicalproperties as a traditional sheet of paper except that it can berewritten many times. The Gyricon technology is essentially a techniqueof manipulating millions of small toner particles in a thin layer oftransparent plastic wherein the toner particles are arranged in responseto an application of voltage patterns. The image displayed by theGyricon sheet will be maintained until new voltage patterns are applied.It should be noted that other flexible display technologies formanufacturing flexible displays may be used, such as organiclight-emitting diode (OLED), organic/polymer thin film transistor (TFT),or any combination thereof.

FIG. 2 illustrates other components that may be part of electronicinteractive device 100, such as actuators 142, 144, 146, and 148, anelectrostatic device 149, a wireless transceiver 160, a touch sensor152, a flex gesture sensor 156, an accelerometer 154, and a controller170.

Actuators 142, 144, 146, and 148 and the electrostatic device 149 may beconfigured to generate a haptic effect at a surface, such as surface110, of electronic interactive device 100. The haptic effect may includea mechanical movement, such as a vibration of device 100 or an actuationof objects like pins or rods that project from a surface to touch auser. The haptic effect may include an electrostatic interaction, eitherto generate a force on an object, like a force that attracts a user'sfinger to surface 110, or to send an electric signal to an object thatcan perceive the signal, like a nerve of the user's finger or a sensorin a stylus. One or more of the actuators may comprise a fiber (ornanotube) of electroactive polymers (EAP), a strip of a piezoelectricelement, a fiber of shape memory alloy (SMA), a voice coil, a magneticactuator such as a solenoid, a pneumatic actuator, an ultrasonic energyactuator, an eccentric mass actuator, or any combination thereof. One ormore of the actuators may be a flexible actuator, such as one formedfrom SMA fibers. The actuators may generate a vibration, lateral motion,up and down motion, rotational motion, or any combination thereof.Haptic actuators are discussed in more detail in co-pending U.S.application Ser. No. 11/758,638, filed Jun. 5, 2007 and entitled “METHODAND APPARATUS FOR HAPTIC ENABLED FLEXIBLE TOUCH SENSITIVE SURFACE,” theentire content of which is incorporated by reference herein. AlthoughFIG. 2 shows four actuators, other embodiments of device 100 may havefewer (e.g., 0, 1, 2, 3) actuators or may have more (e.g., 5 or more)actuators. Each of the actuators may be located at a variety oflocations, such as at an edge, corner, or center of electronicinteractive device 100. The actuators may further have different sizes,or may have the same size.

Electrostatic device 149 may comprise any device that applies voltagesand currents instead of mechanical motion to generate a haptic effect.In some embodiments, electrostatic device 149 may comprise an insulatinglayer over a conductive layer. In one example, the insulating layer maycomprise a portion of flexible surface 110. Electrostatic device 149 mayoperate by applying an alternating-current electric signal to theconductive layer. The signal may capacitively couple the conductivelayer with an object, such as a user's finger or a stylus, near or onthe insulating layer. The capacitive coupling may generate a hapticeffect by varying a frictional force experienced by the object as itmoves across the insulating layer. Haptic electrostatic devices arediscussed in more detail in co-pending U.S. application Ser. No.13/092,269, filed Apr. 22, 2011 and entitled “ELECTRO-VIBROTACTILEDISPLAY,” the entire content of which is incorporated herein byreference.

Flex gesture sensor 156 may be configured to sense a flex gestureapplied to a flexible surface, such as surface 110. In some embodiments,sensing the flex gesture may include sensing a magnitude of the gestureand a direction of the gesture. For example, sensor 156 may beconfigured to sense not only a bending gesture, but also a direction inwhich the flexible surface was bent (e.g., up or down) and a magnitudeof the bending. The magnitude of bending may include a rate at which thesurface is bent, a degree to which the surface is bent, a duration forwhich the surface is bent, or any combination thereof. Sensor 156 mayinclude any device configured to sense a gross deformation in a surface.For example, sensor 156 may include a strain gauge, an accelerometer, agyroscope, a piezoelectric transducer, a pressure sensor, an opticalfiber, or any combination thereof. In some embodiments, accelerometer154 may be part of sensor 156. For example, sensor 156 may comprise astrain gauge and accelerometer that senses a magnitude of a flexgesture. In the example, the strain gauge may sense a degree ofdeformation while the accelerometer may sense a rate of deformation. Inthe example, sensor 156 may further comprise a gyroscope that may sensea direction of the flex gesture by sensing a direction of deformation.

In some embodiments, accelerometer 154 may be separate from anyaccelerometers of sensor 156. Accelerometer 154 may be configured tomeasure, for example, a rate of movement of a person carrying device100. In some embodiments, device 100 may have a gyroscope separate fromany gyroscopes of sensor 156. The gyroscope may be configured tomeasure, for example, an angle at which device 100 is tilted. In someembodiments, any piezoelectric transducer of actuators 142, 144, 146, or148 may be separate from any piezoelectric transducer of sensor 156. Insome embodiments, a piezoelectric transducer of sensor 156 may also bepart of one of the actuators of device 100.

In some embodiments, one or more transducers, such as a piezoelectrictransducer of device 100, may be configured to harvest power from agesture or other movement of the device. In some embodiments, the device100 may have no battery, and power harvested by the one or moretransducers may completely power sensors, wireless transceivers,controllers, and haptic devices of device 100.

Touch sensor 152 may comprise one or more sensors configured to detectan object, such as a user's finger or a stylus, near or touching asurface of electronic interactive device 100, such as a surface ofscreen 130, a surface of user input device 122, or any combinationthereof. In one example, touch sensor 152 may comprise one or morecapacitance sensors embedded in device 100 and may be configured tosense a change in capacitance arising from a user's touch and to sense alocation of the touch. In one example, touch sensor 152 may comprise apressure sensor, a resistive sensor, a surface acoustic wave (SAW)sensor, an infrared sensor, an optical sensor, or any other sensorconfigured to sense a touch input at the surface of device 100. In someembodiments, touch sensor 152 may be part of one or more user inputdevices 122, 124, 126, 128, or screen 130.

Wireless transceiver 160 may be configured to communicate a sensed touchgesture and a sensed flex gesture to, for example, a game console, adesktop computer, a router, a home entertainment system, or anycombination thereof. Wireless transceiver may be configured tocommunicate via a wireless protocol such as IEEE 802.11, Bluetooth®, aninfrared communication protocol, a CDMA protocol, or any other wirelesscommunication protocol.

Controller 170 may be configured to control operations of one or morecomponents of device 100 and to trigger a haptic effect and anapplication action in response to a flex gesture. In one example,controller 170 may be communicatively coupled to user input devices 122,124, 126, and 128 to receive user input received at the devices. In oneexample, controller 170 may be communicatively coupled to screen 130 topresent visual content on the screen and to receive user input if screen130 is a touch screen. In some embodiments, controller 170 may triggerthe application action by presenting the action on screen 130. Forexample, if application action were a zoom action, controller 170 mayzoom visual content presented on screen 130. In some embodiments,controller 170 may be communicatively coupled to touch sensor 152, flexgesture sensor 156, and wireless transceiver 160. In response to agesture sensed by sensor 152 or 156, controller 170 may causetransceiver 160 to transmit a signal associated with the gesture to agame console, which may present an application action associated withthe gesture on a TV screen. In some embodiments, controller 170 may becommunicatively coupled to actuators 142, 144, 146, 148, andelectrostatic device 149 to provide a haptic effect in response toreceiving a sensed gesture. Controller 170 may comprise logic circuitry,one or more processors, or any other computing circuit.

In some embodiments, one or more components of device 100 may beflexible and may form part of a flexible surface, such as surface 110,or may be operatively coupled to the flexible surface.

FIGS. 3-7 illustrate example flex gestures that may be applied toflexible surface 110 of electronic interactive device 100. FIG. 3 showsa bending gesture that may be applied to flexible surface 110. A frontview of the bent surface along a line 3A-3A is shown in FIG. 3A. As theFigure shows, the flex gesture may be applied in a first direction, suchas downward or inward, may be applied in a second direction, such asupward or outward, and may be applied in another direction. In someembodiments, the second direction may be opposite to the firstdirection. In some embodiments, the second direction may be orthogonalto or have some other relationship with the first direction. Forexample, the first direction may relate to bending of a left and rightedge of device 100, while the second, orthogonal direction may relate tobending of an upper and lower edge of device 100. In some embodiments, adeformation may have a magnitude. The magnitude may include a degree ofdeformation, a rate at which the surface was deformed, a duration ofdeformation, or any combination thereof. In some embodiments, bendingsurface 110 to a threshold degree or less may constitute a first flexgesture, while bending the surface beyond the threshold degree mayconstitute a second flex gesture.

FIG. 4 show a twisting gesture that may be applied to one side offlexible surface 110. As seen in FIG. 4A, which shows a front view ofthe twisted surface along line 4A-4A, twisting may be applied to only aleft side of surface 110. As seen in FIG. 4B, which shows a side view ofthe twisted surface along line 4B-4B, the flex gesture may be applied ina first direction, in which a left top corner is flexed outward, awayfrom a user (assuming the surface were facing the user) or downward(assuming the surface were facing up). In the first direction, a leftbottom corner may be flexed inward, toward the user or upward.

As seen in FIG. 5, the flex gesture may be applied in a second, oppositedirection. As shown in FIG. 5B, which shows the side view of twistedsurface 110 along line 5B-5B, the left top corner may be flexed inwardtoward the user or upward and the left bottom corner is flexed outward,away from the user or downward. FIG. 5A shows that the second directionof twisting may be applied to the left side of surface 110. The twistinggesture may be applied to the right side of surface 110 in a similarmanner. In some embodiments, twisting to a threshold degree or less mayconstitute a first twisting gesture, while twisting beyond the thresholddegree may constitute a second twisting gesture.

FIGS. 6 and 7 show a twisting flex gesture that may be applied to twoopposing sides of surface 110. As seen in FIGS. 6A and 6B, the twistinggesture may be applied in a first direction, in which a left top cornerof surface 110 is twisted outward, away from a user or downward and aright bottom corner is flexed outward, away the user or downward. Asseen in FIGS. 7A and 7B, the twisting gesture may be applied in asecond, opposite direction, in which a right top corner is twistedoutward and away from the user or downward and a left bottom corner istwisted outward and away from the user or downward. The flex gesturesshown are only examples, and any other flex gesture may be applied to aflexible surface of an electronic interactive device. For example, aflex gesture may roll flexible surface 110, may bend only a corner ofsurface 110 without twisting the surface, or bend only one side offlexible surface 110.

A flex gesture sensed by the electronic interactive device 100 may beused as a user input that triggers an application action. The action maybe triggered by, for example, controller 170. In some embodiments, theapplication action may be related to actions in a virtual or simulatedenvironment, such as in a game. For example, application actions maycomprise moving forward or backward in the game, firing a weapon orloading a weapon in the game, zooming in or zooming out of a binocularview of the game environment, or shuffling or passing cards in the game.Each of the application actions may be associated with a flex gesture,and may further be associated with a direction, magnitude, or acombination thereof of the flex gesture. For example, a bending gesturein an outward direction may be associated with moving forward in thegame environment, while the bending gesture in an inward direction maybe associated with moving backward in the game environment. A twistinggesture that flexes an upper right corner in an upward direction may beassociated with firing a weapon in the game, while the twisting gesturethat flexes the upper right corner in a downward direction may beassociated with reloading the weapon in the game. In some embodiments,associations between flex gestures and application actions in the gameor any other application may be customized by a user. The customizationmay be performed by, for example, controller 170, or by a computingdevice separate from device 100.

In some embodiments, flex gestures used in games and other applicationsmay be applied with other forms of user input. In one example, a flexgesture may be associated with acceleration or deceleration actions of avehicle in a game, while signals from accelerometer 154 may beassociated with steering actions. In one example, a flex gesture may beassociated with a movement action in a game, while pressing user inputdevice 128 may be associated with weapon firing actions in the game. Inone example, a flex gesture may be associated with weapon firingactions, while a gyroscope in device 100 may measure orientationsassociated with aiming directions of the weapon (or steering directionsof a car) in the game. In one example, a bending gesture may beassociated with drawing a bow in a game, while pressing user inputdevice 124 may be associated with releasing the bow to shoot an arrow.

In some embodiments, an application action may be related to navigatinga menu. In these embodiments, a bending gesture may be associated withadvancing or retreating through a menu, scrolling up or scrolling downthrough menu options, or selecting or deselecting a menu option. Atwisting gesture may be associated with an action to switch betweenmenus for presentation on a screen.

In some embodiments, an application action may be related to controllinga multimedia platform, such as an eBook, music, or video applicationrunning on device 100 or a home entertainment system communicating withdevice 100. In these embodiments, a bending gesture may be associatedwith browsing forward or browsing backward through book pages, mediatracks, or channels. A twisting gesture may be associated with a volumeup or a volume down action. In another example, a flex gesture such asbending a corner of surface 110 may be associated with turning device100 on or turning device 100 off, or with turning a TV on or a TV off.The application actions discussed herein are only examples, and a flexgesture may be associated with any application and any response to auser input.

Although some examples discussed herein show that a flex gesture in twoopposite directions may be associated with two opposite applicationactions (e.g., moving forward and moving backward), the flex gesture inthe two opposite directions may be associated with any two applicationactions. For example, a bending gesture in an upward direction may beassociated with an application action to present a phone dial screen,while a bending gesture in a downward direction may be associated withan application action to play music.

In some embodiments, flex gestures may be combined. For example, anapplication action such as placing a TV controlled by device 100 insleep mode may require a bending gesture in a first direction followedby a twisting gesture in a second direction. The twisting gesture mayhave to be applied while surface 110 is still bent, or may be appliedafter surface 110 is bent and then relaxed. For an application action,such as the TV hibernation action, associated with a sequence of flexgestures, other application actions such as outputting of an audioconfirmation may be provided after one or more gestures in the sequenceto confirm that the sequence is being correctly followed. In anotherexample, a bending gesture may be associated with drawing a bow in agame, while a twisting gesture applied while the surface is bent may beassociated with releasing the bow to shoot an arrow.

In some embodiments, an application action may be associated with asequence having a touch gesture and a flex gesture. For example, abending gesture may be associated with toggling between a phone vibratemode and a phone ring mode on device 100, but controller 170 may choosenot to trigger the application action unless a user has applied a touchgesture to surface 110 within a threshold amount of time before the flexgesture. For example, the user may need to apply a swiping motion in anydirection on screen 130 before bending surface 110. If only the bendinggesture has been detected, controller 170 may give no response or maypresent a visual or audio output indicating that the togglingapplication action cannot be triggered because no touch gesture has beenreceived. By incorporating a touch gesture into the toggle action,device 100 may avoid inadvertently toggling from a vibrate mode to aring mode. In some embodiments, a touch gesture may have to be appliedbefore any flex gesture is able to trigger an intended applicationaction. Applying a touch gesture to a surface may include touching thesurface, making a swiping motion on the surface, making a pinching orexpanding motion on the surface, or any other touch gesture.

In some embodiments, an application action may be associated with asequence having an accelerometer-based gesture and a flex gesture. Forexample, an accelerometer-based gesture such as a swinging gesture maybe made with device 100 and may be associated with simulating a tennisswing for a virtual tennis game. Accelerometer 154 and a gyroscope, forexample, may detect the swing. In response to detection of the swing, ananimation of a tennis swing on a screen may be triggered, such as bycontroller 170. In the example, a flex gesture may be associated with,for example, simulating throwing up a tennis ball for a serve. Flexgesture sensor 156, for example, may detect the flex gesture. Inresponse to detection of the flex gesture, controller 170 may trigger ananimation of a tennis serve on the screen.

Triggering an application action may comprise presenting the applicationaction on a screen or any other user output device (e.g., a speaker). Inone example, controller 170 may present a movement action in a game or amenu selection action on screen 130. Presenting the action may includecausing a video chip in controller 170 to render visual contentcorresponding to the action. In one example, controller 170 may triggera zoom action associated with a flex gesture by causing wirelesstransceiver 160 to transmit to another computing device such as a gameconsole a signal associated with the flex gesture. The game console maythen present the action on a TV screen. In the example, controller 170may cause wireless transceiver 160 to transmit a movement command to thegame console, or controller 170 may cause wireless transceiver 160 tosimply transmit a signal indicating that a bending gesture in an outwarddirection, for example, has occurred. In the latter scenario, the gameconsole may then generate the movement command from the bending gesturebased on a gesture-to-action mapping stored on the console.

In some embodiments, a flex gesture may trigger a haptic effect. Ahaptic effect refers to a stimulus or force, including but not limitedto a vibration, an attractive or repulsive force, a voltage or current,some other mechanical or electromagnetic force, heating or cooling, orsome other stimulus. The haptic effect may comprise one or a combinationof the forces or stimuli described herein. A plurality of haptic effectsmay be combined to form an overall haptic effect. In one example,controller 170 may sense the flex gesture and activate one or more ofactuators 142, 144, 146, 148, and electrostatic device 149. The hapticeffect among flex gestures may be the same, or may differ. For example,controller 170 may activate only actuator 144 in response to a bendinggesture in an upward direction, may activate actuators 142 and 144 inresponse to the bending gesture in a downward direction, and mayactivate all four actuators in response to a twisting gesture. In someembodiments, a type, duration, or intensity of a haptic effect may alsobe varied based on what flex gesture is applied or on what applicationaction is triggered. For example, a haptic effect triggered in responseto a flex gesture that increases music volume may be more intense than ahaptic effect triggered in response to a flex gesture that decreasesmusic volume. If the flex gesture is later associated with a differentapplication action, such as an action to change a TV channel or scrollthrough a menu, a different type of haptic effect (e.g., electrostatic-or thermal-based haptic effect) may then be associated with the flexgesture. In some embodiments, a type, duration, or intensity of a hapticeffect may be controlled by the flex gesture. For example, the intensity(e.g., frequency and/or amplitude) of the haptic effect may be based ona magnitude of the flex gesture (e.g., a rate or degree of flexing). Themore or faster that a flexible surface of an electronic interactivedevice is flexed, such as to trigger a greater acceleration in a game ora faster scrolling in an electronic document, the higher the frequencyor amplitude of the haptic effect may be.

In some embodiments, a haptic effect may be triggered in response to aflex gesture only if the flex gesture is valid. A flex gesture may beinvalid, for example, if it has no application action associated withit. In some embodiments, a default haptic effect (e.g., vibration ofactuator 144 for 2 seconds at intensity level of 3) may be triggered toindicate to a user that the gesture is invalid.

In some embodiments, a haptic effect may be associated with a sequenceof gestures, such as a sequence of a touch gesture and a flex gesture.The sequences may be associated with an application action such as theaction to place a TV in sleep mode or the action to toggle between aphone vibrate mode and a phone ring mode, as discussed above. In oneexamples, a haptic effect (e.g., vibration of actuators 142 and 144) maybe provided only if the sequence of gestures is correctly applied. Adifferent haptic effect (e.g., vibration of actuator 142) may betriggered following one or more gestures in the sequence to confirm thatthe sequence is being correctly followed. In one example, a defaulthaptic effect (e.g., vibration of actuator 144 for 2 seconds atintensity level of 3) may be triggered in response to all invalidsequences of gestures.

Although the invention has been described in detail for the purpose ofillustration based on what is currently considered to be the mostpractical and preferred embodiments, it is to be understood that suchdetail is solely for that purpose and that the invention is not limitedto the disclosed embodiments, but, on the contrary, is intended to covermodifications and equivalent arrangements that are within the spirit andscope of the appended claims. For example, it is to be understood thatthe present invention contemplates that, to the extent possible, one ormore features of any embodiment can be combined with one or morefeatures of any other embodiment.

1-20. (canceled)
 21. An electronic interactive device comprising: a userinterface comprising a flexible surface; a sensor configured to sense afirst flex gesture applied to the flexible surface, and to sense asecond flex gesture that is applied to the flexible surface while theflexible surface is still deformed from the first flex gesture; a hapticoutput device configured to generate a haptic effect; and a controllerin signal communication with the sensor and the haptic output device,wherein the controller is configured, in response to the sensor sensingthe second flex gesture being applied to the flexible surface while theflexible surface is still deformed from the first flex gesture, totrigger an application action associated with the first flex gesture andthe second flex gesture, and to control the haptic output device togenerate the haptic effect.
 22. The electronic interactive device ofclaim 21, wherein the first flex gesture is a bend gesture, and thesecond flex gesture is a twist gesture, such that the controller isconfigured to control the haptic output device to generate the hapticeffect in response to the twist gesture being applied to the flexiblesurface while the flexible surface is still bent from the bend gesture.23. The electronic interactive device of claim 21, wherein theelectronic interactive device is a phone, and the flexible surface has aflexible OLED screen.
 24. The electronic interactive device of claim 21,wherein the controller is configured to control a parameter of thehaptic effect based on a duration of the first flex gesture or of thesecond flex gesture.
 25. The electronic interactive device of claim 21,wherein the controller is configured to control a parameter of thehaptic effect based on a rate of flexing of the first flex gesture or ofthe second flex gesture.
 26. An electronic interactive devicecomprising: a user interface comprising a flexible surface; a sensorconfigured to sense a flex gesture that bends a corner of the flexiblesurface; a haptic output device configured to generate a haptic effect;and a controller in signal communication with the sensor and the hapticoutput device, the controller configured, upon the sensor detecting theflex gesture bending the corner of the flexible surface, to trigger anapplication action associated with the flex gesture, and to control thehaptic output device to generate the haptic effect.
 27. The electronicinteractive device of claim 26, wherein the flex gesture bends only thecorner of the flexible surface.
 28. The electronic interactive device ofclaim 26, wherein the controller is configured to control a parameter ofthe haptic effect based on at least one of a duration or a rate offlexing of the flex gesture.
 29. The electronic interactive device ofclaim 28, wherein the parameter is a frequency of the haptic effect oran intensity of the haptic effect.
 30. The electronic interactive deviceof claim 28, wherein the sensor is a first sensor configured to sensethe rate of flexing of the flex gesture, and wherein the electronicinteractive device further comprises a second sensor configured to senseat least one of a direction of the flex gesture or a degree of flexingof the flex gesture, and wherein the haptic effect is further based onthe at least one of the direction of the flex gesture or the degree offlexing of the flex gesture.
 31. The electronic interactive device ofclaim 30, wherein the first sensor is an accelerometer, and the secondsensor is a strain gauge or a gyroscope.
 32. An electronic interactivedevice comprising: a user interface comprising a flexible surface; asensor configured to sense a flex gesture applied to the flexiblesurface; a plurality of haptic output devices that are configured togenerate haptic effects; and a controller in signal communication withthe sensor and the plurality of haptic output devices, wherein thecontroller is configured, in response to the sensor sensing the flexgesture being applied to the flexible surface: to trigger an applicationaction associated with the flex gesture, and to select a haptic outputdevice from among the plurality of haptic output devices based on atleast one of a type or direction of the flex gesture, and to control thehaptic output device that is selected to generate a haptic effect. 33.The electronic interactive device of claim 32, wherein the controller isconfigured to select the haptic output device based on whether the typeof the flex gesture is a bend gesture or is a twist gesture.
 34. Theelectronic interactive device of claim 32, wherein the controller isconfigured to select only one haptic output device from among theplurality of haptic output devices if the flex gesture is in a firstdirection, and is configured to select more than one haptic outputdevice from among the plurality of haptic output devices if the flexgesture is in a second and opposite direction.
 35. The electronicinteractive device of claim 32, wherein the controller is configured tocontrol a parameter of the haptic effect based on at least one of adegree of flexing of the flex gesture, a duration of the flex gesture,or a rate of flexing of the flex gesture.
 36. The electronic interactivedevice of claim 35, wherein the parameter of the haptic effect is basedon the duration of the flex gesture.
 37. The electronic interactivedevice of claim 35, wherein the parameter of the haptic effect is basedon the rate of flexing of the flex gesture.
 38. The electronicinteractive device of claim 35, wherein the parameter is an intensity ofthe haptic effect.
 39. The electronic interactive device of claim 35,wherein the parameter is a frequency of the haptic effect.